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ABSTRACT 

Researchers from BBN Laboratories and Lesley College 
(Massachus et ts) have begun exploratory studies to build an 
understanding of the learning that occurs when middle school students 
interact with sci ence~based simulations. Sixty hours of discussion 
among eight middle school students were videotaped as they used the 
simulations from the "Physics and Biology Explorer Series" (1992, 
Scotts Valley, California: Wings for Learning). The videotape 
analysis suggests the need for a better understanding of the dynamics 
of mental-model development as middle school students use simulations 
to learn science. The need to better understand the role of prior 
knowledge and the students' ability to use scientific problem-solving 
skills in a modeling environment is also apparent. Four figures 
illustrate the discussion. An appendix presents a proposed Modeling 
Skills Assessment Inventory Checklist (MOSAIC). (Contains 12 
references .) (Author/SLD) 



************************************************* ************** ******** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 

*********************************************************************** 



On 

O 

a 

w 






0«e.°o?E?uMhOTnTn NT ** e00c * T '0* 
EOUCAT/nwAi Resea 'Ch and Improvement 

"^ssr 0 """'™ 

° £l£zr,£’J" en '”'*»*«> « 

originating n ** perSon ° r organization 
'oprodocfion^ua^ty* 6660 made ,0 ,mpr ° w o 






™ent do nornec«, n «? S,a, ° dm,h,$d0Cu - 
OER. Pos.i.on o^^ repreSent *«*•» 



Using Simulations 
for 

Learning Science 



by Nancy Roberts 



Lesley College 
29 Everett Street 
Cambridge, MA 02138-2790 



p- 



o 



w 

v> 




‘ PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 

Nancy Roberts 




BEST COPY AVAILABLE 

£ 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC).” 



Using Simulations for Learning Science 

George Blakeslee and Nancy Roberts 
Lesley College 

Bill Barowy, Lorraine Grosslight, Christine Theberge 
BBN Laboratories 



Introduction/ Abstract 

Researchers from BBN and Lesley College have begun exploratory studies to build 
an understanding of the learning that occurs when middle school students interact 
with science-based simulations. We videotaped 60 hours of discussion among 
middle school students as they used simulations from the Physics and Biology 
Explorer Series. The videotape analysis suggests the need for a better understanding 
of: 

• the dynamics of mental model development as middle school students use 
simulations to learn science, and 

• the role of prior knowledge and interest on students’ ability to use scientific 
problem-solving skills in a modeling environment. 

Background 

Using Simulations for Learning Science is a project to investigate the thinking and 
learning behavior of middle school students. The work is being guided by three 
research questions: 

1. What simulation-specific skills do middle school students possess and 
how well developed are these skills? 

2. What higher-order simulation-related analytical skills do middle school 
students possess and how well developed are these skills? 

3. Can middle school students transfer modeling and simulation skills 
across subject domains? 

Protocol analysis (Chi & Bassok) of videotaped interviews is the primary means of 
data collection. We are taking a social semiotic approach in this study. Lemke 
describes this as asking "how we make sense of and to one another and how we 
make sense of the world." (p. 186) The research begins with a prestudy involving 2 
pairs of seventh grade students, one male and the other female, from a suburban 
town in Massachusetts. 

The pilot clinical interviews did not provide information on either the students' 
general understanding of "models" or scientific models in particular. Consequently 
we modified our procedure and added an initial structured interview developed by 
Grosslight. This interview examines and classifies students’ understandings of 
models. Following the Grosslight interview we conduct the baseline clinical 
interview using the Explorer "Waves" simulation. "Waves" was selected for the 




We worked intensively with 8 seventh grade students from an inner-city, 
Massachusetts school. The students were teacher selected to represent a 
wide spectrum of abilities. Following the Grosslight and Explorer "Waves" 
interviews, we engaged the students in teaching experiments using the 
Explorer "Population Ecology" model to study the concept of equilibrium in 
aquatic populations. The "Population" simulation represents a middle level 
of abstraction. The students with whom we worked had studied ecology in 
science with the aid of a classroom aquarium. However, the Explorer 
representation for the "Population" model is abstract. 
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Figure 2. Screen from the explorer "Population Ecology" simulation 

The students all attended eight hour-long sessions with 4 students present 
each time. The classroom teacher worked with one pair of students while 
one of the researchers worked with the other pair. Each pair of students 
was videotaped at each session. 

The analysis of the "Population Ecology" teaching experiment suggested that the 
teacher's participation and presentation style restricted our ability to elicit model 
related behavior and thinking. The teacher was unable to function in other than a 
directive style. Consequently, we developed a second teaching experiment, focusing 
on the cardiovascular system using the Explorer "Cardiovascular" simulation. The 
"Cardio" simulation is the least abstract of the Explorer series. These students had 
just finished studying the heart and the simulation is an animation of a body 
showing the heart functioning. 







Figure 3. Screen from the Explorer Cardiovascular" similation 



During these sessions we introduced dynamic feedback systems thinking (Roberts) 
and microcomputer based laboratory data collection along with the simulation. We 
modified our teaching style to allow maximum student expression and 
investigation. The same 8 Massachusetts middle school students participated in the 
"Cardiovascular" teaching experiment. We regrouped them into 2 groups of four in 
order to study in more depth 3 boys and 1 girl who each responded uniquely and 
differently to the simulations. These four students also represented a wide range of 
subject knowledge and abilities. In order to reduce the effect of the teacher and 
school setting, this group was brought to BBN Laboratories for their sessions. 



Investigation 



The project was conducted in three overlapping phases. Phase 1 is the baseline 
clinical interview designed to establish maximum student engagement with the 
computer model and minimum student-interviewer interaction. We use the 
Explorer "Waves" model and initially employ a "cold introduction" approach 
which consists of showing the students the salient features of the Macintosh 
computer and the "Waves" model. The "Waves" model was chosen because the 
topic is not included in the school’s elementary or middle school curricula and 
therefore not formally studied by the students. However, it is a phenomenon they 
might have had experience with in a variety of other ways. We are interested in 
how the students go about exploring this novel computer simulation and how they 
bring to bear previous experience. To elicit model related thinking from the 
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students we ask them to explain, out loud, their reaction to and reasoning about the 
computer program (we avoided using the word model ) as they explore it. 

By the end of the fourth interview we realized that middle school students find the 
"Waves" model to be unique, but not necessarily interesting. Subsequently, when 
interest waned, we asked leading questions about two screen buttons — "graph" and 
"controls". We also developed an intervention approach, when needed, that 
consists of presenting the students with a challenge to make the model behave in 
some particular way. This intervention proved helpful in differentiating students’ 
levels of operational skill. 

Phase 2 consists of a multi-session teaching experiment in which the students use a 
simulation model to learn science. We begin this phase with the Grosslight 
structured interview about models. The use of the Grosslight interview allows us to 
categorize our subjects' knowledge level about models and permits us to connect 
our work to related modeling research. 

The teaching experiment was designed to coincide with a unit on population 
ecology then being studied in school. This permited us to explore the relationship 
between subject specific knowledge and model related thinking skills. Through 
presenting students with challenges we test the extent to which student knowledge 
of population ecology can be made operational in a computer simulation setting. 

Phase 3 is an expanded teaching experiment of a specific science content area — the 
cardiovascular system — that the students had already studied in school. We 
introduce two new thinking tools: causal-loop (feedback) thinking and 
microcomputer based laboratory (MBL) data collection. This phase began by asking 
the students what they know about the behavior of the heart and circulatory system. 
Having studied the subject in school, the students show a fair degree of systemic 
knowledge and take very readily to expressing this understanding from a feedback 
perspective using causal-loop diagrams. 

We next presented the students with an exercise scenario and asked them to predict 
how the cardiovascular system behaves and how we might test their predictions. 

We introduce microcomputer based laboratory tools and encourage the students to 
design experiments to test their predictions. Finally we introduced them to the 
Explorer "Cardiovascular" simulation and challenged them to replicate the 
experiment using the computer simulation. The students then compared and 
debated the results. 

Preliminary Results 

The preliminary results of this study suggest two lines of investigation. One is the 
need for a better understanding of the dynamics of mental model development as 
middle school students use computer simulations to learn science. The second is the 
need to better understand the role of prior knowledge and interest on students' 
ability to use scientific problem-solving skills in a modeling environment. 
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Protocol analysis (Chi & Bassok) of all the videotapes has been conducted to identify 
both the dynamics of mental model development and the appearance of model 
related, as well as general thinking and problem solving, skills. The preliminary 
data suggest that middle school students learn science with computer simulation 
models along three dynamic dimensions: 1) as a unique, possibly novel, 
phenomenon; 2) by comparing the model to prior experience; and 3) as a model of a 
real phenomenon. See figure 1. 
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Figure 4. Dimensions of Student Learning 

In addition, this work has produced a preliminary version of a proposed Modeling 
Skills Assessment Inventory Checklist (MOSAIC). See Appendix. This instrument 
is based on the type developed by Loucks & Crandall (1982). 

Using Simulations for Mental Model Construction 



The results of this preliminary study suggest that middle school students define 
models primarily as physical constructions or recognizable illustrations. This 
process is referred to by Sutton as "figuring," applying "something we already have 
in order to make sense of the relatively unfamiliar" (p. 1217) Grosslight catorgorized 
these students as level one in their understanding of the concept "model". Models 
are generally understood to be created for demonstration or educational purposes. 

We found that middle school students can and do learn science with computer 
simulation models. The process is complex. It involves three interrelated sets of 
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modeling skills given a dynamic, hands-on learning environment and appears 
strongly influenced by prior experience, interest, and the teaching-learning context. 



Controlled Exploring (see #1 on figure 4) 

We observe three sets of modeling skills in middle schools students during the 
clinical interviews and teaching experiments. First, a student's initial experience 
with a computer simulation model requires a set of general exploratory learning 
skills. These skills must be transferred from other learning experiences because, 
until sufficient information has been elicited from the model’s behavior to 
categorize the experience, the student is faced with a new phenomenon. The 
computer simulation models we use display numerical, graphical, and 
representational images. Therefore the exploratory skills needed by students are 
those of variable control and numerical and graphical interpretation. 

We observe a continuum of variable manipulation and numerical and graphical 
interpretation skills among our middle school students ranging from random 
probing, which a RAND study of students interacting with microworlds labeled 
"thrashing" (McArthur and Lewis), to classic variable control. At this stage students 
frequently refer to the model with pronouns and use variable names as reference 
labels (Sutton) rather than functional concepts. The following excerpts from 
transcripts of experimental sessions illustrate this continuum of skills. 

Exploring a computer model as a novel event . 

An example of vidoe game/ random testing . George is the researcher. 

George: "What have you changed?" 

Nathanielle: "The frequency." 

George: "Do you have a guess as to what you think is going to happen to 
the wave?" 

Nathanielle: "It's gonna go higher." 

George: "Why do you think it's going to go higher?" 

Nathanielle: "Because more frequency. I don't know what it means but..." 

An example of random testing with one, but not the other, of the students able to 
transfer the skills of controlling variables . Bill is the researcher. 
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Susan: "She changed all of them [variables], and if you wanted to see what 
the difference was then you keep only one variable because then you 
don't know what's causing it to be different." 

Bill: "Is that something you've learned here?" 

Susan and Jill: "Yeah, in science." 

and a few minutes later: 

Susan: "So let's not fool around with the 40 [amplitude] since we have a 
good guess. Let's try one we don't know about like friction." [sets 
friction up to mid point] 

Jill: "But lets see if it [amplitude] goes to 30 this time [is changing friction 
value at same time] 

Susan: "But you don't want to test two things at the same time because 
then you don't know what's causing it [sets amplitude back to 40 and 
runs simulation]. It's kind of wider, more subtle, smoother, not as 
extreme — I'm not positive though. Let me try again ... it's not going up 
to 40 ... that's because we changed [increased] the friction. The higher 
the friction, the less height it goes. 

Example of testing using the extreme case: 

Brione changed two of the variables by putting them in the middle to see 
what would happen. 

Nathanielle and Brione change the other two variables by putting them in 
the middle too. Brione increases frequency and friction while 
decreasing number of snaps and the amplitude. A straight line results. 
Through trial and error the students are trying to achieve one wave on 
the screen with not much luck yet. The students increase all the 
numbers and get many waves. 

Nathanielle: "How about we take off the friction, put the friction on zero 
and leave the rest up?" 

Brione puts the number of snaps [snaps create the energy source for the 
waves] all the way up and puts everything else on zero. Nathanielle 
tries to tell him it will still just give them a straight line, but Brione 
disagrees because he put the number of snaps up. Nathanielle was 
right. They get a straight line. Now they try what Nathanielle 
suggested — friction at zero and everything else up. They get many 
waves. 
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Another example of using the testing of extremes as a trial and error 
strategy rather than as a modeling strategy . 



Nancy sets up for line graph trial by setting variables to the largest value 
she thinks will be accepted. 

Nancy has no idea why the line graph comes out as it does. She giggles and 
then suggests changing the frequency. 

Bill shows them how to use the variable slider bar. 

"Lets move them [variables] as far as they will go [up]." 

Bill asks why they are setting all the variables to their upper limit. 

Nancy: "I want to see what happens." 

Nancy couldn't make any sense of the resulting screens. 

Nancy: "Let's put them [variables] as low as they go and see what 
happens." 

Peggy: "We put them as far down as they go and that's what [pointing to 
wave animation] it's doing [no amplitude]. So now we put them in the 
middle and we're going to see what happens." 

An example of random trial and error that leads to some conclusions about 
computer models . The following transcript is from the same interview and shows 
Nancy beginning to see the model's consistency: 

Bill: "Is there anything you haven't figured out about the controls 
[variables]?" 

Nancy: "No, nothing. We've done them all." 

Bill then asks about each variable and gets the following answers: 
amplitude - "Makes it higher and lower" 

frequency - "Makes it skinny and fat"; then relates amplitude and 
frequency "skinny makes it go higher; fat lower" 
snaps - "gives it [wave] a push. If you just give it three [snaps] it 
doesn't go as far." 

friction - "don't know - we really haven't done much with it." 

Another example of students' beginning to notice the model's consistent behavior: 

George asks what they are thinking. The program is running and a wave 
moves across the screen. 
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George: "Have you ever seen anything like that before?" 

Brione: "Yes, just a minute ago." 

Comparing Prior Experience to Model (see #2 in figure 4) 

A second set of modeling skills are seen when students have had prior contact with 
the target phenomenon — hands-on experience or formal study — and recognize the 
computer simulation model as an example of a class of phenomena, rather than as a 
unique event. We observe a continuum of skills in these situations that range from 
simple recognition to functional replication. The student's exploratory skills are 
employed at this stage to compare and match the computer simulation's behavior 
with prior experiences. The student's language now includes analogy, as described 
by Clement, to prior experience and the use of prior experience to explain the 
model's behavior. We also observe model-controlled reasoning based on the 
discovery of consistent model behaviors — what we have come to call "video game" 
behavior. 

An example of students matching the behavior of the simulation to a 
prior experience (The word "wave" is not used initially by the 
researcher and does not appear on the computer screen): 

George explores what they know about the words: 

amplitude "size of the wave" (only David has heard it before but 
didn't know what it meant); 

frequency - David: "number of waves by the speed they travel at." 

Colin: "yeah"; snaps - "number of times the wave snaps down" (flicks 
his wrist for number of snaps) 

David: "The number of times it is going to go like this [makes 
undulating motion through the air with his hand]. 

But the students' knowledge is not necessarily immediately operational: 

George gives Colin & David a challenge to test if students are really 

relating the simulation to waves: "Use the controls and buttons to put 
three waves on the screen." 

Colin: "Oh, just to put three waves on screen ... that's easy." 

David sets friction to maximum, amplitude to middle, frequency to 
middle, snaps to 3 and starts the simulation. 

Both students count number of snaps: "1, 2, 3." 

David: "See it goes down." [Friction is dampening leading waves.] 
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Colin: " That one almost disappears. We've got to lessen the friction. 
Count that one out. Don't put no amplitude. It's going too slow. You 
have to raise the amplitude." 

David: "Do you know what you're doing?" 

Colin: "Yes, I do. 100% amplitude; frequency - what does frequency do - oh 
yeah, how wide they are, 20%; number of snaps, 3. We got to less down 
the friction to 2." 

Colin : "There - 1,2. Do they all have to be on at the same time?" 

George: "Yes." 

Colin: "3, aw that one disappeared." 

George: "I would like to see 3 waves on the screen at the same time. What 
would you change to make it do that? Do you think you're close?" 

Colin: "Yeah, lower the amplitude." 

Colin misunderstood the role of amplitude. Frequency changes the width 
and thus the number of waves shown on the screen. 

Example of students connecting simulation to a prior related idea . This 
behavior is similar to what Wong points out as "finding a particular 
analogy is closely associated with getting the correct answer." (p. 1271) 

Nathanielle: "It looks like it's going on forever. It's not gonna stop. I think 
that makes it go longer — friction." 

George: "Does that make any sense to you?" 

Brione: "Yeah, because I know that if you use friction, friction makes 
things stop. So if there's less friction or no friction then it goes on for a 
long long time." 

George: "Where have you used friction before?" 

Brione: "I haven't used it but I've talked and heard of it in one of my 
science classes." 

Examples of students connecting to other experiences with wave 
phenomena: 

George asks what they are thinking. The program is running and a wave 
moves across the screen. 
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George: "Have you ever seen anything like that before?" 

Brione: "Yes, just a minute ago." 

Nathanielle: "The water." 

George: "Like water?" 

Brione: "Yeah, it flows like water actually it does. Even though it's not 
water it flows like water." 

Nathanielle: "The waves and the ocean." 

George: "Is that exactly like water you see?" 

They both say no. 

Brione: 'The water I've seen keeps on going and going.' 

George: 'And what's this doing?' 

Brione: "It seems to stop." 

Nathanielle: "It's slowing down. Does it ever stop?" 

George: "Could you find out?" 

Nathanielle: "Yeah" 

l 

George: "How could you find out?" 

Brione: "Cause see how it stays there?" 

Nathanielle: "It never stopped." 

George: "Have you ever seen a wave that stops?" 

Brione: "Yeah, it stops when it hits the ground, like when 
it hits the shore but another one comes again." 

Brione: "It looks like one of those heart beat things." 

Nathanielle: "Yeah, like at the hospital when someone's in critical 
condition. (They all laugh) They're about to die the lines going 
straight." 
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Another example of students connecting the simulation to prior 
experiences. 

Bill: "Click on the running man, that will make it go. Why don’t you try 
doing that." [Ocean waves] 

John: "Yeah, it sort of looks like it." 

Bill "Does it sort of seem the same to you?" 

Paul: "Yeah, it looks like waves." 

John: "Like museum of science. That thing." 

Bill "Where have you seen that?" 

John: "In Boston, at the Museum of Science. That simulator thing." 

Paul: "Like a wavemaking machine." 

Bill: "Does it act just like that [wave model]?" 

Paul: "I don't think they are that high. They're mostly the same [moving 
hand in wave like fashion] where that [wave model display] has one 
big one and the others are kind of smaller and it [museum of science 
wave machine] has like a big board that goes back and forth and the 
waves are all the same basically." 

Bill: "Why do they do that [have a wave machine]?" 

Paul: "Its like simulation to show you how things work." 

Bill "How do they work? Have you ever thought about that before?" 

Both boys: "No, not really." 

Paul: "It 7 s kind of like a rope where the loop keeps falling through and 
getting smaller and smaller kind of like that [wave model display]. You 
flip it like that and the loops keep getting smaller to the end like that. 

Bill: "I was wondering, have you noticed any other differences or things 
the same?" 

Paul: "Not really. There is not really a pattern — they're all different sizes 
except sort of like a heart monitor. It goes up and down like that except 
when the heart stops and it's a straight line. That thing [snaps] is like 
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with the rope. You make it go up and down once except the wave 
doesn't come back." 

Bill: "Now click on controls." 

John: "It might be a radio wave now that I see all this [points to variables]. 
Frequency and Amplitude refer to radio waves and they’re sort of like 
that [points to wave model display] too." 

Bill: "Frequency and Amplitude? Where have you seen that mentioned?" 

John: "AM and FM radio." 

Bill: "Radio is waves then?" 

Both boys: "Yeah." 

Paul: "Friction is like the rope - it gets smaller and is not going as far. The 
rope must be tighter - suppose it was a rope. So it won' t go as 

far - it doesn't have enough slack. 

Bill: "Is this like anything you’ve seen?" 

Paul: "Do you mean the wave like? I've done it with my garden hose 
when I'm going to water the lawn and it gets too tight. So you have to 
loosen it up and do like that [makes flipping motion with hand] to get 
it over." 

Bill: "So you flip it and it goes over and what happens? Does it keep 
going?" 

Paul: You make the loop like that [points to initial large wave on the 
display] and it gets smaller and smaller and you hope it will be high 
enough to get over." 

Behavior Testing (see #3 in figure 4) 

A third set of modeling skills appear when students have developed a functional 
understanding (mental model or concept) of the target phenomenon. The computer 
simulation is accepted as a reasonable representation of the concept. We observe a 
continuum of skills at this point ranging from concept confirmation to behavior 
prediction. Students begin to use the computer simulation model to test ideas and to 
propose hypotheses. Their language changes to use the model's behavior to explain 
observations of the target phenomenon or to justify predictions of the target 
phenomenon's behavior. 
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The next set of transcripts, part of eight lessons on population dynamics in a 
simulation environment of fish, daphnia, and algae, illustrate how students’ skills 
and concepts develop in a teaching/leaming environment. The more formal study 
creates a context in which variable control skills improve and ability to replicate 
computer model behavior is acquired through experience. Pam is the classroom 
teacher. 

An example of using the behavior of the model to explore a content 
concept. Consistent model behavior can be identified and the 
implication that the computer model results can be applied to the 
target phenomenon. 

Bill notices that Nancy figures out a setting that keeps everything alive 
and stable. He shows them the numbers she comes out with (12 fish 
and 25 daphnia) and reminds them that they will have only 5 fish in 
their aquariums. 

Katie wants to halve all the present numbers to equal 6 fish and 15 
daphnia. Nancy resets the model. The girls go on to try other small 
numbers such as 3 fish. They discuss how to figure out the closest 
ratios. 

Nancy resets the variables to those they decide are closest to 5 fish and rim 
their test. They are both glued to the screen counting the rise in the 
population of fish. 

Nancy has been changing the initial amounts and finds that they always 
come back to the same ending amounts. 

Trying a new set of ratios: 

Nancy: "Just to see how it works, right?" 

The model comes out to 12 fish again. 

Bill and Katie: "Why?" 

Bill: "Because it's set up to be a bigger aquarium. You can try changing the 
habitat support as long as you keep it the same for all of them." 

Bill asks for a prediction. 

Katie: "If all habitats are set lower, then it should be OK with fewer fish." 

(Katie constantly moves about, scratching head with pencil, playing 
with hair, doodling on notebook.) 

New settings are 4 fish and 7 daphnia. 



0 



17 



15 



■ J- 



Nancy counts as fish multiply. Again they end up with 12. Lower habitat 
more fish. They try again. 

Nancy: "6 fish, 5 fish, 4 fish, thank you so much! Five, eight, and 46, it's 
not changing! [To Katie] Do you have this written down? Let me do it. 

You weren't listening to me. 

And another example . 

David: "We should experiment ..." 

Brione shows David & Bill the bottle aquarium. 

David: "We need to find out how many this can support. We can only put 
in about 10 fish. I've got it leveled off." Then describes ratio of fish to 
daphnia to algae." 

Brione asks about changing habitat support. 

David gives answer with 2 conditions: one why you shouldn't change; one 
why you should. "We'll be playing with their lives." 

Brione: "What does habitat support mean in real terms?" 

David: "Do they have enough to eat, room to swim, ..." 

Pam: "Was having the aquarium here helpful?" 

David: "The computer is the basic tool." 

Brione: "It takes a while for 2 fish to have more fish." 

Last 3 minutes show that work with simulation has generated a lot of 
questions about the real aquarium. 

The following transcripts are from the last set of lessons using the "Cardio" model 
chosen because the students had just finished studying the heart in school. 

An example of students who have a well developed mental model of a 
phenomenon — dynamic behavior can be diagrammed and causal 
reasoning used to predict and explain expected results of both 
experimental activities and computer model results. This reinforces 
Schwartz’s findings on the power of visualization. Nancy R. is a 
researcher. 
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Nancy R: "Have you studied the heart and what it's function is in the 
body?" 

Nathanielle: "Sort of, we just started to." 

Bill: "You just started to in school with Ms. Hunter?" 

Nancy: "Yeah." 

Bill: "What have you learned so far?" 

Nancy: "We learned about a pacemaker." 

Bill: "What is a pacemaker?" 

Nancy: "It um performs the beats in your heart." 

Nathanielle: "That the veins lead to the heart and about the arteries." 

Nancy: "The arteries, capillaries, the red blood cells and the white blood 
cells." 

Bill: "So veins, arteries and capillaries I can never keep them separated, 
what's the difference?" 

Nancy: "The veins lead blood to the heart. The arteries take it away from 
the heart." 

Nathanielle: "Away from the heart." 

Bill: "Away from the heart?" 

Nancy: "The capillaries — they pick up from the where the arteries and 
bring it to the veins and the veins bring it back." 

Bill: "Okay." 

Nancy: "I can draw a picture better." 

Nancy R: "Do you have any idea of what the heart does, the function of 
the organ?" 

Nancy: "It like takes blood in and pumps it out." 

Nancy R: "Pumps it out — and do you have any idea why the blood has to 
go throughout the body?" 
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Nancy: "So you can move." 

Nancy: "We worked with stethoscopes last week." 

Bill: "Did you?" 

Nancy: "We took our pulses and did exercises." 

Nathanielle: "We did jumping jacks." 

Nancy: "And took our pulses and running in place and walking in place." 

Bill: "Yeah but what happened to everybody else?" 

Nancy: "Their heart beat got faster." 

Bill: "Their heart beat got faster? Does that make any sense? Is that a 
reasonable thing to do?" 

Nancy: "Yeah, because if you move faster then your heart beat gets faster." 

Bill: "Okay, if you move faster then your heart beat goes faster, but yeah 
why should it work that way?" 

Nancy: "Because movement makes your heart beat, it has to do things 
faster if your doing things faster." 

Bill: "So you guys have some very interesting data, it seems. You got pulse 
rates, before and after exercise. You have some different exercises and 
you found that the pulse rates changed. Did you discuss any of that in 
class?" 

Nancy: "Not really, she gave us questions to do but I don’t think that 
anybody did them." [They giggle.] 

Bill: "That's great, I would like to discuss some of that now. I would like to 
try and make some sense of what that is, of why that data is the way it 
is. An I'm interested in any ideas that you have. Have you guys 
thought about it?" 

Nancy: "Can I go to the board?" 

Bill: "Go right ahead. I'd like to see. [Nancy goes to the board and draws 
the diagram below] Wow there's something we haven't thought of." 
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Nancy R: "Now tell us, talk through it." 

Bill: "Wait, she's still drawing. Okay, so tell us what you got so I can figure 
out." 

Nancy: "I don't know. I think if you exercise and then your heart beat goes 
faster then you breathe faster and that tires you out." 

Bill: "Okay, so that affects how tired you are." 

Nancy R: "And so why does that cause you to do less exercise when you 
start breathing quickly?" 

Nancy: "Yeah it causes you to do no exercise." 

An example of students comparing the computer "Cardio" simulation graphs to 
their predictions (generated from their own experiments of running, resting, and 
recording their changing pulse rates graphed). 

Bill: "Do you remember what we were doing last time? We were making 
Brione run and we got some data." 

Nancy: "Um huh, and I proved the theory." 

Bill: "What was that?" 

Nancy: "That no matter if you run, walk, or run and sit, your heart rate 
will come down at the same time and it did. It took 45 seconds for 
both." 
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Bill: "So what we're interested in doing this time is doing a simulation. 

We have a simulation of the cardiovascular system. Okay, and one of 
the things we can look at is heart rate. Alright, um, so one of the 
things I want to show you if you guys grab the mouse there are a couple 
things to show you. There are buttons over on the left. If you click on 
those they cause our simulated man to either be resting, running or 
walking. Okay and it's like he's been doing that for a long time. So if 
you hit walking rate I want you to notice his heart rate is somewhere 
around 78 beats /minute. And if you hit running rate it is somewhere 
around 129." 



Bill: "How do you guys think — does it seem possible to do the experiment 
that we were doing before this way?" [using the simulation] 

All: "Ah huh." 

Bill: "What do you think we have to do now to test?" 

Nancy: "Go from running to walking and then go from running to resting 
and see how long it takes." 

Nancy: "I'm going to do running to walking." 

David: "So far it's taken this guy 77 seconds to slow his heart rate down to 
a little bit over walking. I'm still waiting for it to go down to 57." 

Bill: "Okay." 

Bill: "What kind of results are you getting? Are they anything like what 
you thought you would get?" 

Nancy: "It takes a lot longer to go from running to resting, mine is almost 
down there and it's only been 51 seconds." 

Nancy R: "Is that what you had predicted Nancy last week?" 

Nancy: "No last week we had 45 seconds for both so." 

Nancy R: "So last week your prediction was that it would take the same 
amount of time?" 

Nancy: "Yeah, so far that one's taken a lot longer." 

Bill: "Which one’s taken longer?" 
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Nancy: "That one — 126 seconds and mine's almost down there, it's only 
56 seconds." 

Bill: "And you're doing running to walking and Peter you're doing 

running to resting and it's taking longer to go from running to resting 
than it is to go from running to walking. Okay." 

Nancy R: "So which do you think is right, your predictions from last week 
or the computer simulation? Which do think is more acurate?" 

Nancy: "The simulation." 

Bill: "The simulation, why?" 

Peter: "Because then we don't have to run again." [everyone laughs] 

Nancy: "What if it's not accurate. Don't ask him a question, please." 

David: "The simulation is better because you could do the heart beat of 
three or four different people and with our experiments you can..." 

Peter: "But also it seems right because it seems that if it's going down 
further it should take longer." 

Nancy: "It does seem right." 

Notice the change in Nancy's behavior from her interaction with the waves 
simulation, the aquarium unit, and finally the cardiovascular unit. Initially we 
assumed Nancy had no modeling skills on any of the dimensions we delineated. 
However, during the cardiovascular sessions Nancy emerged from a background 
position to a leader in this investigation. We are proposing that because Nancy would 
like to become a doctor and therefore enjoyed and found relevant her classroom 
lessons on the heart, she could muster all the necessary modeling skills during these 
last sessions to behave as a scientist. We do not believe that during her six hours 
exposure to these models before we introduced "Cardio" to the group that Nancy could 
have learned modeling in an operational way. We believe her total change in 
personality is closely related to her prior knowledge and interest now coming to bear 
(Clement, Wong). As Norman puts it, "People's views of the world, of themselves, of 
their own capabilities, and of the tasks that they are asked to perform, or topics they are 
asked to learn, depend heavily on the conceptualizations that they bring to the task." 

(p. 7) 

Conclusions 

The three stages of model related skill use are not independent of one another. Each 
successive stage builds on those before, specifically the recognition and replication 
dimension calls upon exploration skills, and the confirmation and prediction 
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dimension requires replication skills. As students' experience with the 
phenomenon and the computer simulation model grow their mental model 
develops through the three stages represented by the three axes of figure 1. It appears 
that until the computer model comes to be understood as a representation of the 
target phenomenon the simulation will not be used in a scientifically meaningful 
way by the student. 

This paper represents one view of the data generated by the three student- 
simulation interactions — "Waves", "Population", and "Cardio". The research team 
is looking at the tapes from several other perspectives to gain additional insights 
concerning the nature of the interactions of science simulations and middle school 
students. 



Appendix 

A proposed Modeling Skills Assessment Inventory Checklist (MOSAIC) 

MOSAIC is designed to identify major categories of model-related thinking 
skills along the three dimensions of computer simulation-facilitated mental 
model development shown in figure 1. These categories emerged from the 
videotapes. The researchers' hope is that this inventory will inform other 
researchers /teachers on students current modeling abilities. 

1. Controlled Exploring 

• Model Internal Skills - those needed to arrive at an understanding of the 
behavior of a model, and to determine it's consistency, and logic. 

• Model External Skills - those needed to compare a model against 
observed data and to determine it's validity and complexity. 

2. Comparing to Prior Experiences 

• Model Interpretation Skills - those needed to interpret and infer 
meaning from numerical and graphical displays. 

• Model Recognition and Replication Skills is currently missing from 
MOSAIC 

3. Behavior Testing 

• Model-Related Thinking Skills - those needed to understand cause and 

effect, feedback, change over time, and to consider alternative versions 
of the modeled system. 

• General Modeling Issue Skills - those needed to understand the value 

and limitation of a model, to derive meaning from animated screen 
images, to differentiate the model from experience, and to develop an 
understanding of the relationship between a model and the target 
phenomena. 

MOSAIC - Modeling Skill Assessment Inventory Checklist 
Controlled Explorating 

1. "Model-internal skills" - Exploring a model 

Skills to arrive at an understanding of the behavior of a model. 

Posing questions and exploring a model by: 
a) isolating variables 
Single 

1. none 

2. one (then another/no reset) 

3. one (then another/reset) 

4. one (various settings) 

Sets 
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5. not all or set 

6. all or set (random) 

7. all or set (pattern) 

8. all or set (with reason) 

b) testing limits 

1. none 

2. possible setting among many 

3. trial & error "just to see" 

4. methodical checking 

c) examining interrelationships between parameters. 

1. none 

2. notice, not used 

3. notice pattern 

4. methodical checking 

d) controlling parameters, 

1. none 

2. accidental 

3. repeat pattern (video game) 

4. set with reason 

Testing a model to see if its behavior is consistent and logical under a wide 
variety of conditions by: 

a) collecting evidence 

1. none 

2. recollection of settings & screen action 

3. explore with different parameters 

4. explore with one parameter 

b) looking for patterns in the model behavior 

1. none 

2. describe screen appearance 

3. correlate screen action with parameter setting 

4. express conceptual reason for screen action 

c) correlating model displays 

1. none 

2. accidental 

3. screen correlation (video game) 

4. express conceptual reason for correlation 

2. "Model-external skills" - Skill in determining model validity by comparing model 
with target phenomenon 

Comparing the model against observed data 

a) assessing the applicability of the model to the data 

1. no connection 

2. connection (rejected) 

3. connection by analogy 

b) analyzing analogy for reliability, reasonableness, and fit. 

1. confirming, analogy semi-quantitative 
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» a. not explored 

b. explored 

c. predictions 

2. confirming, analogy quantitative 

a. not explored 

b. explored 

c. predictions 

3. discontinuing, analogy semi-quantitative 

a. not explored 

b. explored 

c. predictions 

4. disconfirming, analogy quantitative 

a. not explored 

b. explored 

c. predictions 

Skills in determining the complexity of the model. 

a) Propose the phenomena be modeled in a simpler fashion? 

b) Propose critical elements of the system are missing and need to be 
included in the model? 

Comparing to Prior Experiences 

3. Model Interpretation Skills 

a) Reading and interpreting graphs, and using graphs as tools for 
expressing and identifying behavior. 

1. recognize 

2. decode - mechanical 

3. decode - meaningful 

4. reason with meaning 

b) Drawing inferences from data. 

1. none 

2. recall past model behavior 

3. about model with model 

4. about model with experience 

5. predictions 

c) Using basic arithmetic reasoning to infer relationships. 

1. none 

2. accidental 

3. decode - mechanical 

4. decode - meaningful 

5. reason with meaning 

d) Using and interpreting statistical information. 

e) See patterns in the underlying structure of a problem. 

4. Model Recognition and Replication Skills missing from our MOSAIC 
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Testing Behavior 



5. Model-Related Thinking 

a) Identifying a complex system - Quantitative 

Identifying a complex system - Semi-Quantitative (Qualitative) 

b) Identifying in a complex system: 

1. cause-and-effect - Quantitative 

cause-and-effect - Semi-Quantitative (Qualitative) 

2. feedback - Quantitative 

feedback - Semi-Quantitative (Qualitative) 

3. change over time - Quantitative 

change over time - Semi-Quantitative (Qualitative 

c) Grappling with alternative explanations for the same 

phenomenon 

1. making a reasoned decision about which to support or where further 

exploration is needed 

2. conducting further exploration 

6. General Modeling Issues 

a) Shows an understanding of both the value and limitations of a model. 

b) Considers the meaning derived from animated screen images. 

c) Differentiate between the model and experience. 

d) Considers the relationship between a model and target phenomenon 
and processes and expresses a sense of how the model explains the 
phenomenon. 
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